A Comparison of Head Movement Classification Methods.

To understand human behavior, it is essential to study it in the context of natural movement in immersive, three-dimensional environments. Virtual reality (VR), with head-mounted displays, offers an unprecedented compromise between ecological validity and experimental control. However, such technological advancements mean that new data streams will become more widely available, and therefore, a need arises to standardize methodologies by which these streams are analyzed. One such data stream is that of head position and rotation tracking, now made easily available from head-mounted systems. The current study presents five candidate algorithms of varying complexity for classifying head movements. Each algorithm is compared against human rater classifications and graded based on the overall agreement as well as biases in metrics such as movement onset/offset time and movement amplitude. Finally, we conclude this article by offering recommendations for the best practices and considerations for VR researchers looking to incorporate head movement analysis in their future studies.

What the Flip? What the P-N Flip Can Tell Us about Proactive Suppression.

It has been debated whether salient distractors in visual search can be proactively suppressed to completely prevent attentional capture, as the occurrence of proactive suppression implies that the initial shift of attention is not entirely driven by physical salience. While the presence of a Pd component in the EEG (associated with suppression) without a preceding N2pc component (associated with selection) has been used as evidence for proactive suppression, the link between these ERPs and the underlying mechanisms is not always clear. This is exemplified in two recent articles that observed the same waveform pattern, where an early Pd-like component flipped to a N2pc-like component, but provided vastly different interpretations (Drisdelle, B. L., & Eimer, E. PD components and distractor inhibition in visual search: New evidence for the signal suppression hypothesis. Psychophysiology, 58, e13898, 2021; Kerzel, D., & Burra, N. Capture by context elements, not attentional suppression of distractors, explains the PD with small search displays. Journal of Cognitive Neuroscience, 32, 1170-1183, 2020). Using RAGNAROC (Wyble et al., Understanding visual attention with RAGNAROC: A Reflexive Attention Gradient through Neural AttRactOr Competition. Psychological Review, 127, 1163-1198, 2020), a computational model of reflexive attention, we successfully simulated this ERP pattern with minimal changes to its existing architecture, providing a parsimonious and mechanistic explanation for this flip in the EEG that is unique from both of the previous interpretations. Our account supports the occurrence of proactive suppression and demonstrates the benefits of incorporating computational modeling into theory building.

A Case for Studying Naturalistic Eye and Head Movements in Virtual Environments.

Using head mounted displays (HMDs) in conjunction with virtual reality (VR), vision researchers are able to capture more naturalistic vision in an experimentally controlled setting. Namely, eye movements can be accurately tracked as they occur in concert with head movements as subjects navigate virtual environments. A benefit of this approach is that, unlike other mobile eye tracking (ET) set-ups in unconstrained settings, the experimenter has precise control over the location and timing of stimulus presentation, making it easier to compare findings between HMD studies and those that use monitor displays, which account for the bulk of previous work in eye movement research and vision sciences more generally. Here, a visual discrimination paradigm is presented as a proof of concept to demonstrate the applicability of collecting eye and head tracking data from an HMD in VR for vision research. The current work's contribution is 3-fold: firstly, results demonstrating both the strengths and the weaknesses of recording and classifying eye and head tracking data in VR, secondly, a highly flexible graphical user interface (GUI) used to generate the current experiment, is offered to lower the software development start-up cost of future researchers transitioning to a VR space, and finally, the dataset analyzed here of behavioral, eye and head tracking data synchronized with environmental variables from a task specifically designed to elicit a variety of eye and head movements could be an asset in testing future eye movement classification algorithms.

I tried a bunch of things: The dangers of unexpected overfitting in classification of brain data.

Machine learning has enhanced the abilities of neuroscientists to interpret information collected through EEG, fMRI, and MEG data. With these powerful techniques comes the danger of overfitting of hyperparameters which can render results invalid. We refer to this problem as 'overhyping' and show that it is pernicious despite commonly used precautions. Overhyping occurs when analysis decisions are made after observing analysis outcomes and can produce results that are partially or even completely spurious. It is commonly assumed that cross-validation is an effective protection against overfitting or overhyping, but this is not actually true. In this article, we show that spurious results can be obtained on random data by modifying hyperparameters in seemingly innocuous ways, despite the use of cross-validation. We recommend a number of techniques for limiting overhyping, such as lock boxes, blind analyses, pre-registrations, and nested cross-validation. These techniques, are common in other fields that use machine learning, including computer science and physics. Adopting similar safeguards is critical for ensuring the robustness of machine-learning techniques in the neurosciences.

Understanding visual attention with RAGNAROC: A reflexive attention gradient through neural AttRactOr competition.

A quintessential challenge for any perceptual system is the need to focus on task-relevant information without being blindsided by unexpected, yet important information. The human visual system incorporates several solutions to this challenge, 1 of which is a reflexive covert attention system that is rapidly responsive to both the physical salience and the task-relevance of new information. This article presents a model that simulates behavioral and neural correlates of reflexive attention as the product of brief neural attractor states that are formed across the visual hierarchy when attention is engaged. Such attractors emerge from an attentional gradient distributed over a population of topographically organized neurons and serve to focus processing at 1 or more locations in the visual field, while inhibiting the processing of lower priority information. The model moves toward a resolution of key debates about the nature of reflexive attention, such as whether it is parallel or serial, and whether suppression effects are distributed in a spatial surround, or selectively at the location of distractors. The model also develops a framework for understanding the neural mechanisms of visual attention as a spatiotopic decision process within a hierarchy and links them to observable correlates such as accuracy, reaction time (RT), and the N2pc and PD components of the electroencephalogram (EEG). This last contribution is the most crucial for repairing the disconnect that exists between our understanding of behavioral and neural correlates of attention. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Abrupt darkening under high dynamic range (HDR) luminance invokes facilitation for high-contrast targets and grouping by luminance similarity.

When scanning across a scene, luminance can vary by up to 100,000-to-1 (high dynamic range, HDR), requiring multiple normalizing mechanisms spanning from the retina to the cortex to support visual acuity and recognition. Vision models based on standard dynamic range (SDR) luminance contrast ratios below 100-to-1 have limited ability to generalize to real-world scenes with HDR luminance. To characterize how orientation and luminance are linked in brain mechanisms for luminance normalization, we measured orientation discrimination of Gabor targets under HDR luminance dynamics. We report a novel phenomenon, that abrupt 10- to 100-fold darkening engages contextual facilitation, distorting the apparent orientation of a high-contrast central target. Surprisingly, facilitation was influenced by grouping by luminance similarity, as well as by the degree of luminance variability in the surround. These results challenge vision models based solely on activity normalization and raise new questions that will lead to models that perform better in real-world scenes.

A delay in sampling information from temporally autocorrelated visual stimuli.

Much of our world changes smoothly in time, yet the allocation of attention is typically studied with sudden changes - transients. A sizeable lag in selecting feature information is seen when stimuli change smoothly. Yet this lag is not seen with temporally uncorrelated rapid serial visual presentation (RSVP) stimuli. This suggests that temporal autocorrelation of a feature paradoxically increases the latency at which information is sampled. To test this, participants are asked to report the color of a disk when a cue was presented. There is an increase in selection latency when the disk's color changed smoothly compared to randomly. This increase is due to the smooth color change presented after the cue rather than extrapolated predictions based on the color changes presented before. These results support an attentional drag theory, whereby attentional engagement is prolonged when features change smoothly. A computational model provides insights into the potential underlying neural mechanisms.

A 100,000-to-1 high dynamic range (HDR) luminance display for investigating visual perception under real-world luminance dynamics.

BACKGROUND: Real-world illumination challenges both autonomous sensing and displays, because scene luminance can vary by up to 109-to-1, whereas vision models have limited ability to generalize beyond 100-to-1 luminance contrast. Brain mechanisms automatically normalize the visual input based on feature context, but they remain poorly understood because of the limitations of commercially available displays. NEW METHOD: Here, we describe procedures for setup, calibration, and precision check of an HDR display system, based on a JVC DLA-RS600U reference projector, with over 100,000-to-1 luminance dynamic range (636-0.006055 cd/m2), pseudo 11 bit grayscale precision, and 3 ms temporal precision in the MATLAB/Psychtoolbox software environment. The setup is synchronized with electroencephalography (EEG) and infrared eye-tracking measurements. RESULTS: We show display metrics including light scatter versus average display luminance (ADL), spatial uniformity, and spatial uniformity at high spatial frequency. We also show a luminance normalization phenomenon, contextual facilitation of a high contrast target, whose discovery required HDR display. COMPARISON WITH EXISTING METHODS: This system provides 100-fold greater dynamic range than standard 1000-to-1 contrast displays and increases the number of gray levels from 256 or 1024 (8 or 10 bits) to 2048 (pseudo 11 bits), enabling the study of mesopic-to-photopic vision, at the expense of spatial non-uniformities. CONCLUSIONS: This HDR research capability opens new questions of how visual perception is resilient to real-world luminance dynamics and will lead to improved visual modeling of dense urban and forest environments and of mixed indoor-outdoor environments such as cockpits and augmented reality. Our display metrics code can be found at https://github.com/USArmyResearchLab/ARL-Display-Metrics-and-Average-Display-Luminance.

A hierarchical model of visual processing simulates neural mechanisms underlying reflexive attention.

The visual system deploys attention reflexively in response to important stimuli to facilitate rapid selection. To better understand how a system that is distributed across a hierarchy of brain areas coordinates this rapid response, we have developed a computational model that simulates reflexive attention as a recurrent lock-on state between topographically aligned cortical areas. This model provokes key questions about reflexive attention that we study through the N2pc, an EEG component that indexes the timing and lateralization of stimulus-driven attention. A key finding from such research is that for two sequential targets sharing a location, the second elicits no N2pc, despite being easily perceivable. This suggests that attention locks-on to a target's location and carries forward in time to a second target without being redeployed. Here, we assess key properties of reflexive attention in four EEG experiments. First, the lock-on N2pc effect generalizes to feature target types. Second, the lock-on state is specific to a location, not just a visual hemifield. Third, reflexive attention decays as the interval between two targets increases, such that a second target will once-again elicit an N2pc at longer intervals. Finally, the lock-on state is not specific to a particular target type, which implies that reflexive attention is mediated using representations that are not stimulus-specific. These results provide important constraints on our understanding of visual attention. We incorporate those constraints in a formalized model of attention that elucidates the link between neural mechanisms and a key neural correlate of attention. (PsycINFO Database Record

How do reaching and walking costs affect movement path selection?

Although reaching and walking are commonly coordinated, their coordination has been little studied. We investigated decision-making related to reaching and walking in connection with a recently discovered phenomenon called pre-crastination-the tendency to expend extra effort in the service of hastening goal or sub-goal completion. In the earlier studies where pre-crastination was discovered, participants decided which of two buckets to carry to the end of a walkway, picking the bucket they thought was easier. Surprisingly, the majority of participants chose to carry the bucket that was closer to the start position, which meant that the bucket they chose had to be carried farther than the bucket they did not choose. Here we inquired into participants' sensitivity to reaching effort and walking effort by varying how far participants had to reach to pick up a bucket, how heavy the bucket was, and how far participants had to walk with the bucket they chose. We found that participants were willing to lean and reach far to pick up an empty bucket that was a shorter walk from the start position. However, as reaching costs and carrying costs increased, participants prioritized shorter reaches over shorter walking distances. The results show that although pre-crastination is a robust tendency, there are limits to the kinds of costs people are willing to incur to complete sub-goals as soon as possible.

Non-singleton colors are not attended faster than categories, but they are encoded faster: A combined approach of behavior, modeling and ERPs.

The visual system is able to detect targets according to a variety of criteria, such as by categorical (letter vs digit) or featural attributes (color). These criteria are often used interchangeably in rapid serial visual presentation (RSVP) studies but little is known about how rapidly they are processed. The aim of this work was to compare the time course of attentional selection and memory encoding for different types of target criteria. We conducted two experiments where participants reported one or two targets (T1, T2) presented in lateral RSVP streams. Targets were marked either by being a singleton color (red letter among black letters), being categorically distinct (digits among letters) or non-singleton color (target color letter among heterogeneously colored letters). Using event related potential (ERPs) associated with attention and memory encoding (the N2pc and the P3 respectively), we compared the relative latency of these two processing stages for these three kinds of targets. In addition to these ERP measures, we obtained convergent behavioral measures for attention and memory encoding by presenting two targets in immediate sequence and comparing their relative accuracy and proportion of temporal order errors. Both behavioral and EEG measures revealed that singleton color targets were attended much more quickly than either non-singleton color or categorical targets, and there was very little difference between attention latencies to non-singleton color and categorical targets. There was however a difference in the speed of memory encoding for non-singleton color and category latencies in both behavioral and EEG measures, which shows that encoding latency differences do not always mirror attention latency differences.